1. Field of the Invention
The present invention relates to the insertion of virtual graphics into a video stream such that these graphics may appear to be behind other graphics or complex colored objects in the video stream and/or in front of other complex colored objects or background structures in the video stream.
2. Related Art
Current occlusion schemes typically model consistently colored backgrounds with a single color range or a collection of color ranges. For many sporting applications, foreground objects may be distinguished from the consistently colored backgrounds based on these color ranges. Chroma keying techniques to handle occluding foreground objects in front of the consistently colored backgrounds are well known in the art, and are described, for example, in U.S. patent application Ser. No. 09/734,709, filed on Dec. 13, 2000, now U.S. Pat. No. 7,015,978, which is incorporated by reference herein in its entirety.
For cases where consistently colored backgrounds are not available, one approach that may be used is to place a logo without occlusion in a region of the scene where occluding objects are unlikely. This may lead to less than ideal placement from a visibility standpoint, and may still be susceptible to the occasional occlusion failure of projectile objects. Furthermore, there maybe failure due to overlap with other graphics which are likewise positioned in the region of the scene where occluding objects are unlikely.
Alternately, spatial patterns or warping of capture images have been proposed to model backgrounds with complex detail as described in U.S. patent application Ser. No. 08/662,089, filed on Jun. 12, 1996, now U.S. Pat. No. 5,953,076, which is incorporated by reference herein in its entirety. A key is formulated by comparing the modified background model with the input video, using a pixel by pixel comparison such used in background subtraction. Unfortunately, this approach tends to have difficulties due to alignment issues, and is particularly problematic when the background varies such as in front of a crowd or moving display.
Furthermore, post-production approaches such as roto-scoping may be employed, where as the boundary between an object and the background is defined though manual operations. This is a time consuming approach operationally with application that are limited to pre-produced content.
The present embodiments will now be described with reference to the accompanying drawings. In the drawings, like reference numbers may indicate identical or functionally similar elements.